Global Journal of Computer Science and Technology

Data security has become a major issue in most network protocols. For wireless system, security support is even more important to protect the users as well as the network. Due to this importance, different protocol were designed & deployed with network standards in order to add the security. The security sub layer of IEEE 802.16 employs an authenticated client/server key management protocol in which the B.S, the serve, control the distribution of keying materials to the client M.S. This paper analyzes the physical layer threat & MAC layer threat of WiMAX .First give an overview of security architecture of mobile WiMAX network, then investigate different security vulnerability & gives possible solution to overcome them. These vulnerabilities are the possibilities to forge key messages in Multiand Broadcast operation, some unauthenticated messages which are susceptible to forgery and the unencrypted management communication which reveal important management information. We modify DH key exchange protocol to fit it into mobile WiMAX network as well as eliminate existing weakness in original DH key exchange protocol. Also RSA & Elliptic curve Diffie Hellman key agreement algorithm are discuss which can be used to generate symmetric key between M.S & B.S. Several one way function are presented by using cryptography, which can be used to solve shared key vulnerability in Multi-&Broadcast service. We find the initial network procedure is not effectively secured that makes Manin-the-middle attacks & Denial of service attack possible. Keywords-IEEE 802.16e security, multi-and broadcast service, shared key vulnerability, Key agreement, DoS